Method and apparatus for digital media management, retrieval, and collaboration

ABSTRACT

The software according to the invention incorporates a glossary management tool that makes it easy for each client to customize terminology to the needs of a particular business. With this tool, termed a glossary manager, a company can customize a number of feature names in the system to provide a more familiar context for their users. A system administrator can also customize the manner in which “thumbnail” or “preview” images are presented. The system performs clustering on search queries, and searches media records multi-modally, using two or more approaches such as image searching and text searching. An administrator can tune search parameters. Two or more streams of metadata may be aligned and correlated with a media file, facilitating later searching. The system evaluates itself. It folds popularity information into rankings of search results.

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation of international applicationnumber PCT/US01/26841 designating the United States, filed Aug. 28,2001, incorporated herein by reference, which claims priority from U.S.application No. 60/228,837, filed Aug. 28, 2000, now expired,incorporated herein by reference.

BACKGROUND OF INVENTION

[0002] Creative designers in various fields work with materials frommany different sources when they develop an advertisement, film,brochure, or other finished product. These materials may have beencreated originally by the creative designer, but for a differentproject, and stored in a media library or archive for later reuse. Theymay have been created by colleagues, and stored in an area that allowsdifferent individuals to collaborate on materials. Or they may have beencreated by professional artists for licensing or sale. These materialsare often called rich media files, or assets. Rich media files mayinclude, but are not limited to, video, photography, graphics, audio,mixed media files, logos, presentations, and text. These media files canexist in the widest possible range of formats. It is a ponderous task tomanage such assets. Annotation mechanisms for such assets include thesystem described in U.S. Pat. No. 5,493,677, assigned to the sameassignee as the assignee of the present invention. Systems to managesuch assets include that described in U.S. Pat. No. 6,012,068.

[0003] It would be desirable to have a management system for digitalmedia which streamlines the task of accounting for rights to use suchmedia, including copyright rights. Systems relating to rights managementinclude those described in U.S. Pat. Nos. 4,337,483, 5,201,047,5,260,999, 5,263,158, 5,319,705, 5,438,508, 5,629,980, 5,765,152, and5,553,143.

[0004] Users in different businesses may use different terminology torefer to the various media management functions. For example, some mayuse the term library, while others use archive. Some may use projectworkspace, while others use share or collaboration tool. In manysystems, changing terminology requires tedious programming effort whichrisks introducing errors into the software. It would be desirable tohave a management system that conveniently permits non-technical usersto customize such terminology on a per-system basis. U.S. Pat. No.5,850,561 describes a glossary construction tool for creating glossaryfrom text.

[0005] Different companies using the software according to the inventionmay have different corporate culture, image, and system context withinthe company. It would be desirable to have a management system thatconveniently permits non-technical users to customize the software on aper-system basis with respect to such corporate concerns.

[0006] In a typical asset management system, users browse through mediafile collections and view thumbnail images of files to decide whichfiles they want to work with. These thumbnails are browseables, or smallrepresentations of the actual images, videos, or other media files inthe system. A browseable is created by optimizing an image or videoframe for online browsing, so a browseable has lower resolution andsmaller dimensions than the original file. It is commonplace, however,to find that the resolution and dimensions are not well suited to thecompany. It would be desirable to have a management system thatconveniently permits a system administrator to customize the software inthis respect.

[0007] Natural language processing (NLP) techniques are well known,including their use in information retrieval applications (Strzalkowski,1993), (Strzalkowski, Perez Carballo and Marinescu, 1995), (Evans andZhai, 1996). Past systems have attempted to improve upon vocabularymanagement techniques, for example as described in U.S. Pat. Nos.5,251,316 and 6,125,236. Past approaches for searching multimediainclude U.S. Pat. Nos. 6,243,713 and 5,794,249.

[0008] Clustering is well known, for example in U.S. Pat. No. 5,317,507,5,758,257, 5,675,819, 5,778,362, and 5,875,446. See also Buckley, Chris,J. Walz, M. Mitra and C. Cardie, “Using Clustering and Super Conceptswithin SMART: TREC 6”(http://trec.nist.gov/pubs/trec6/t6_proceedings.html); Zamir, Oren, O.Etzioni, Madani, and Karp, KDD “Fast And Intuitive Clustering Of WebDocuments;” and Koller, Daphne, and Mehran Sahami, ML “HierarchicallyClassifying Documents Using Very Few Words.” Rankings relating torelevance are discussed in U.S. Pat. No. 5,642,502.

[0009] The evaluation of information retrieval systems became anessential part of the field in the early '90s, and was strongly advancedby the TREC evaluations designed at NIST beginning in 1993. The TRECevaluation contains different tracks, but the tracks all share thefollowing common features:

[0010] They are designed to provide a comparative evaluation betweendifferent systems, usually provided by different participants.

[0011] The evaluation is done using strict test conditions that containa set of queries, a collection of documents, and relevance judgements.

[0012] The evaluations use evaluation scores such as precision andrecall that supposedly predict real users' satisfaction from a system.

[0013] While these evaluations are indeed helpful in comparing theperformance of different IR systems, they do not provide constantfeedback on the performance of a live IR system. The base performance ofan IR system could be at first evaluated using a standard measurementsuch as the one above, but as more media files are added to a system andusers submit queries in an uncontrolled manner, it is hard to predict orestimate the performance of the system. In addition, if the system doesnot fall into the initial TREC evaluation tracks, it is necessary todevelop an independent test case—a very costly task. It is desirable tohave a better self-evaluation system for such a digital asset manager.

[0014] Known annotation-related systems are discussed in U.S. Pat. Nos.5,600,775, 6,006,241, and 5,938,724.

SUMMARY OF INVENTION

[0015] The software according to the invention incorporates a glossarymanagement tool that makes it easy for each client to customizeterminology to the needs of a particular business. With this tool,termed a glossary manager, a company can customize a number of featurenames in the system to provide a more familiar context for their users.A system administrator can also customize the manner in which“thumbnail” or “preview” images are presented. The system performsclustering on search queries, and searches media records multi-modally,using two or more approaches such as image searching and text searching.An administrator can tune search parameters. Two or more streams ofmetadata may be aligned and correlated with a media file, facilitatinglater searching. The system evaluates itself. It folds popularityinformation into rankings of search results.

BRIEF DESCRIPTION OF DRAWINGS

[0016] The invention will be described with respect to a drawing inseveral figures, of which:

[0017]FIG. 1 shows a typical rights management editor interaction;

[0018]FIG. 2 shows a typical customization interaction screen;

[0019]FIG. 3 shows a screen permitting customers to adjust searchparameters;

[0020]FIG. 4 shows a sample of a face-matching system according to theinvention; and

[0021]FIG. 5 illustrates a viewer according to the invention with textannotation box.

DETAILED DESCRIPTION

[0022] The inventive software manages media files, or assets, making itpossible to store and retrieve them, to collaborate with colleagues toedit and work with them, and to deliver them, either for furtherprocessing or in finished form for use.

[0023] The workflow of the inventive software is designed to provideflexible choices among features that support a creative user who workswith rich media files. As soon as a user logs on to the software (anoptional step), she can immediately see if any system-internal messagesare waiting in her system email inbox, either sent from other userscollaborating with her on media files, or sent automatically by theinventive system to notify her of various occurrences that may pertainto her work, such as a project update. The user can read her messages,reply to a message, or compose a new message to send to another systemuser. A composed message may consist of text only, or it may have asystem-internal file attached.

[0024] The user may wish to work on a project, such as an advertisingcampaign. She enters the project work area, and creates a new project.This creates a virtual workspace in the inventive system that isdedicated to holding media files and all their related files, includingedited versions, colleague comments, official approvals, and so on. Shedecides which users will be allowed to share access to the projectspace. She may upload files that she has already created into theproject work area. From there, she can send a media file off to acolleague for comments, editing, or approval.

[0025] If she decides to rework a previously created file for use in thenew ad campaign, she can search for the file in the file storage area ofthe inventive system. She enters search terms, generally either bytyping or by speaking into an input device, and finds an appropriatemedia file. She then saves the file to her new project. She may decide,instead, to have the full, high-resolution version of that file sent toher for a different project. In the inventive system, delivery optionsfor high-resolution files include on-line delivery via ftp, http, orspecial network; via CD delivery, or even via mail delivery of the filein analog format. Ordering and file delivery in the inventive system aresubject to several conditions, which the software verifies and tracks.These conditions include whether the user has the proper rights andpermissions, not only to view the file but also to use or publish it. Ifuse is subject to a fee, the fee must be paid, whether by billing,credit card, or other means. If usage requires personal approval, therequest is routed to the right person and approval is granted andtracked, all within the inventive software system.

[0026] Rights Management.

[0027] If a user determines that she needs to use a rights-restrictedmedia file, she must engage in a dialogue with the inventive softwareabout her planned use of the file. Companies can customize sets ofquestions that the software presents to users, depending on which file auser wants to order. The user may have to answer whether she intends topublish the file, where it will be published, to what size audience, atwhat location, on what date. The answers determine whether the file canbe made available, and can also determine the price. For example, if sheis showing ten seconds of a movie for a one-time training seminar inCanada for an audience of fifty people, her cost may be far less thanmaking those same ten seconds of movie available on the home page of amajor entertainment Web site. And making the entire movie available onthe Web may perhaps not be permitted at any price.

[0028] When users order files in the software according to theinvention, the rights management process controls every aspect of theorder. A company may allow its employees to use its media files forpersonal use, marketing use, or in an internal organization project. Orthey may allow all three uses in different situations. They can trackthe user's use of the files, they can designate the licensing type forthe files and make sure a rights agreement is included, and they canenforce legal or company policy restrictions that may apply. The rightsmanagement process helps clients establish all of these elements so theycan apply them appropriately to their stored media files.

[0029] The invention described here makes it easier for individualclients to customize a rights management setup so that the software willmake the right pricing and availability decisions for each projecteduse. Central to the system is a question-answering paradigm: anadministrator or a manager decides what questions should be presented toeach potential user of a file, and what set of multiple choice answersto supply. The software maps the questions and answers to a rights andusage package, and, if applicable, each package is assigned pricing.

[0030] For example, an administrator may determine that a file'savailability depends on:

[0031] What department is giving the presentation?

[0032] Who is the audience?

[0033] Will the image be included in any handouts?

[0034] Will the image be displayed on an overhead?

[0035] When is the presentation date?

[0036] Does the work contain images of any sculpture, painting, drawing,or other art object that may be copyrighted?

[0037] The questions, in turn, are grouped appropriately for a company'sparticular file collections, and are assigned to either groups of filesor individual files. Questions and their answers may serve asfundamental criteria for whether a file is made available; they mayserve as modifiers to an available file by raising or lowering pricingdepending on usage; or they may simply be informational questions thatdo not play a role in decision-making, but allow companies to track fileuse and user needs.

[0038] The Rights Management Process.

[0039] When ordering files, users answer questions that ask forinformation about how they are planning to use the files they order. Thequestions a user answers are the usage questions a company designatesfor each possible use. This set of usage questions is called a rightspackage. Creating a rights package is the first step in the rightsmanagement process.

[0040] A rights package, or set of questions, is added to other rightspackages, to make up a rights package group. This allows a company tomake more than one rights package available for a group of files. Arights package group is added to a particular license type. The licensetype describes the possible ways a file can legally be used. A rightsagreement is added to define details clearly about the user's rights. Inthe final step, the rights agreement is associated with a particularlicense type through a control table. The control table also containspricing and approval information for each rights package.

[0041] The core piece of this process is the creation and grouping ofquestions that cover a company's business and legal interests for theirstored media file usage. Part of this process is completed with what maybe termed a rights package question role editor. A typical editorinteraction is shown in FIG. 1.

[0042] Glossary Customization Tool and Usability Customization Tool.

[0043] The software according to the invention incorporates a glossarymanagement tool that makes it easy for each client to customizeterminology to the needs of a particular business. With this tool,termed a glossary manager, a company can customize a number of featurenames in the system to provide a more familiar context for their users.

[0044] The system also includes a usability management tool thatcustomizes the software interface “look and feel” to more closelyreflect a company's corporate culture, image, and to reflect the systemcontext within the company. A part of the system termed a usabilitymanager allows a company to change system background colors, fonts, andthe embedded logo. This tool also makes localization into a differentlanguage simpler. A typical customization interaction screen is shown inFIG. 2.

[0045] User-Configurable Browseable Generation.

[0046] In a typical asset management system, users browse through mediafile collections and view thumbnail images of files to decide whichfiles they want to work with. These thumbnails are browseables, or smallrepresentations of the actual images, videos, or other media files inthe system. A browseable is created by optimizing an image or videoframe for online browsing, so a browseable has lower resolution andsmaller dimensions than the original file. The browseable (the proxy ofthe actual media file) is what the user sees in the system according tothe invention, and browseable size is generally set by a systemadministrator for use throughout the system. Importantly, the systempermits generating these lower-resolution proxy files in a way that canbe configured by the customer.

[0047] Natural Language Processing Techniques.

[0048] In the system according to the invention, the natural languageprocessing component is layered. The system architecture supports aconventional search algorithm, which operates as a plug-and-play supportapplication. The algorithm is then used in a word expansion process. Theword expansion is applied to both a user's search query, and to thestored files' captions, that is, descriptive metadata identifyingindividual files.

[0049] When a user submits a search query, the query undergoes severaltypes of NLP processing (detailed below), and the result of each elementin the processing adds new query components (for example, synonyms)and/or weights (which emphasize or dewords depending on their value tothe query). The resulting, or processed, query contains expanded termsand weighting information that can be passed to any search engine. Thismakes it possible to use multiagent (or “federated search”) applicationsas well.

[0050] To expand a user search query, once a user query is received, itis divided into individual tokens, which may consist of single words ormultiwords. (Multiwords are words the search engine recognizes as oneunit, instead of as separate words. With this feature, when a usersearches for sea lions, the system does not bring up lions by theocean.) For this process, a variation of conventional pattern matchingis used. If a single word is recognized as matching a word that is partof a stored multiword, the processor decides whether to treat the singleword as part of a multiword, based on the contents of the stored patternand the input pattern. Stored patterns include not just literal words,but also syntactic categories (such as adjective or nonsemanticcategories (such as nationality or government entity), or exact matches.If the input matches the stored pattern information, then it isinterpreted as a multiword rather than as independent words.

[0051] Processing then determines whether the resulting word is afunction word (closed-class) or content word (open-class). The processorignores function words. For content words, the related concepts for eachsense of the word are retrieved from the semantic net. If the root wordis unknown, the word is treated as a keyword, requiring an exact match.

[0052] The system then matches the expanded, weighted search query withstored media files in order to return search results to the user. Mediafiles are identified by the metadata associated with them, e.g. in acaption. Textfiles can be identified by metadata associated with them,and also by their content. When users import media files into thesystem, the software provides a workflow to add captions to each file.The files are then uploaded into the database, and the file captions areexpanded and the information stored for later file searching.

[0053] To expand the file captions, the processor looks up each word inthe caption or text. Words that may be related in the semantic net arelocated based on stored links, and the looked-up word, along with anyrelated words, are all displayed as the “expansion” of that word. Theprocessor determines whether the current word or phrase corresponds to aproper name, a location, or something else. If it corresponds to a name,a name expansion process is invoked that displays the name and relatednames such as nicknames and other variants, based on a linked name file.If the current word or phrase corresponds to a location, a locationexpansion process is invoked that accesses a gazetteer and displays thelocation and related locations based on linked location informationfound in the gazetteer and supporting files. If the current word orphrase is neither a name nor a location, it is expanded using thesemantic net links and weights associated with those links. Stronglyrelated concepts are given high weights, while more remotely relatedconcepts receive lower weights, making them less exact matches. Thus,for a query on car, documents or metadata containing car and automobileare listed highest, followed by those with sedan, coupe, andconvertible, and then by more remotely related concepts such astransmission, hood, and trunk.

[0054] Once the expansion is complete, the expanded word or phrase isstored in an index database, where it is available for use in searchingas described below. Processing then returns to expand the next word orphrase in the text or caption.

[0055] The system according to the invention handles certain words andphrases individually. Multiwords are matched as a whole unit, and namesand locations are identified and looked up in the separate name andlocation files. Next, noun phrases and other syntactic units areidentified. A part-of-speech tagger uses linguistic and statisticalinformation to tag the parts of speech for sections of the user query.Only words that match by part of speech are considered to match, and iftwo or more parts of speech are possible for a particular word, it istagged with both. After tagging, word affixes (i.e. suffixes) arestripped from query words to obtain a word root, using conventionalinflectional morphology. If a word in a query is not known, affixes arestripped from the word one by one until a known word is found.

[0056] An intermediate query is then formulated to match against thefile index database. Texts or captions that match queries are thenreturned, ranked, and displayed to the user, with those that match bestbeing displayed at the top of the list. In an exemplary system, thesearching is implemented by first building a B-tree of ID lists, one foreach concept in the text database. The ID lists have an entry for eachobject whose text contains a reference to a given concept. An entryconsists of an object ID and a weight. The object ID provides a uniqueidentifier and is a positive integer assigned when the object isindexed. The weight reflects the relevance of the concept to theobject's text, and is also a positive integer.

[0057] To add an object to an existing index, the object ID and a weightare inserted into the ID list of every concept that is in any wayrelevant to the text. For searching, the ID lists of every concept inthe query are retrieved and combined as specified by the query. Since IDlists contain IDs with weights in sorted order, determining existenceand relevance of a match is simultaneous and fast, using only a smallnumber of processor instructions for each concept-object pair.

[0058] Search Technologies.

[0059] The system allows users to search for media files with manydifferent types of search queries. For example, users may submit searchqueries by speaking them, typing them, copying them, or drawing them.

[0060] The process of locating a particular file in a large archive is aspecial area for innovation within the inventive software. Files arecharacterized in several ways. First, they have an identifier, generallysimilar to a filename, which is unique within the system and makes itpossible to link up all the objects related to a file. These can includethe actual high-resolution asset, lower-resolution thumbnails or otherproxies for browsing, and information about the file, or metadata.Searching can be performed on the file identifier, or it can beperformed on the metadata. In the case of metadata searching, it isdesirable to offer search alternatives that go beyond the exact matchingprocess involved in a standard keyword search.

[0061] Some systems use controlled vocabulary searching as anoptimization of keyword searching. Keyword searches simply match exactlyon any word in the user's search query that appears in the searchtarget. (In the system according to the invention, the search target isthe metadata describing a media file.) The set of potential keywords isquite large (as large as the vocabulary of English, or whateverlanguage(s) are being used). If there are no limitations on the searchvocabulary that can be employed, a user can enter a search for puma andfail to find any files captioned as mountain lion or cougar, even thoughthey all refer to the same thing. Controlled vocabulary is an attempt toaddress this problem, albeit at considerable cost. In a controlledvocabulary retrieval system, cataloguers all agree to use the sameterms. In practical terms, this implies that, when cataloguing, theymust check their controlled vocabulary lists and be sure not to deviate.Sometimes tools can be built to aid in this process, depending on thesize of the controlled vocabulary. Similarly, tools can also be providedto searchers to control their search requests. However, controlledvocabulary systems do not scale beyond a few thousand terms, since it isimpractical to look up every word in English for every search. Forbroader retrieval systems, for faster cataloguing, and for simplersearching, a different approach is superior.

[0062] In addition to standard keyword and Boolean searching, the systemsoftware incorporates additional advanced technology for locating storedfiles. Rather than limiting searching to a controlled vocabulary, thesystem software includes natural language search, which allowscataloguers and users to employ any words in English (or whatevernatural language the retrieval system is using).

[0063] Natural language search incorporates:

[0064] a semantic network of concepts

[0065] additional linguistic techniques, including:

[0066] phrase matching

[0067] derivational morphology, in lieu of stemming

[0068] part of speech tagging

[0069] name recognition

[0070] location recognition

[0071] User-Tunable Search Parameters.

[0072] The system according to the invention provides a screen forcustomers to adjust search parameters, to reflect their company use ofstored media file collections. This is shown in FIG. 3. While theparameters may themselves be well-known in a searching system, what isemphasized here is that the user (or, more likely, an administrator) canbe granted access to such fundamental decisions about search as:

[0073] (a) how good a match has to be before it is displayed to theuser, e.g. 50%, and

[0074] (b) how “creative” the search should be, i.e. how much should thesearch terms be expanded to include more distant synonyms and relatedterms.

[0075] It should be borne in mind that the system according to theinvention can be carried out on an internet, meaning an IP-basednetwork, and in particular may be carried out on the Internet, meaningthe global IP-based network.

[0076] Multimodal Search.

[0077] Currently, search methods focus on textual input. The currentinvention incorporates new search techniques, and combines them in novelways.

[0078] Image search is becoming useful in commercial applications. Inthe system according to the invention, user search input is provided ina new way. Users may wish to select an existing image as example input,so that a search consists of “Give me more images like this.” Perhapseven more useful is the ability to select part of an image, analogouswith “Give me more like this part.” In the system according to theinvention, identifying the part may be done in either of two exemplaryways:

[0079] 1. Touch screen: user touches the screen to identify the portionof the image that feeds into the search.

[0080] 2. Markup, using pen or other screen drawing metaphor, includingthrough the system media viewer, which is described in more detailbelow.

[0081] addition, search modalities can be combined. This novel approachto search is particularly applicable to multimedia. Examples ofcombined, or multimodal, searches, include:

[0082] voice and text

[0083] touch screen and text

[0084] drawing and voice

[0085] drawing and touch screen

[0086] and so on.

[0087] Vocabulary Management.

[0088] Ideally, the semantic net of concepts is quite large and attemptsto incorporate every word or term in English (or other language beingused for cataloguing and searching). No matter how large the semanticnet may be, there will be a periodic need to expand or edit it. Newwords appear in English periodically, and, although many may be slangand therefore not particularly important in a business context, somewill be real new words and will be important enough to include. Forexample, rollerblading and in-line skating are relatively new terms inEnglish, and depicting those actions is useful in advertising. So theterms need to be added to the semantic net. Semantic net/vocabularymaintenance is generally a manual process, particularly where the userhas an existing media library with a thesaurus and vocabulary managementprocess. Such maintenance can also be performed automatically.

[0089] To maintain a vocabulary for an information retrieval applicationthat accepts user queries in natural language, a user maintaining asemantic net would track search queries in a query log. From the querylog, he would determine which words are actually novel and arecandidates to be added to the system vocabulary, by expanding the-querylog using morphology, and possibly a spell checker and name identifier.The remaining terms that were not matched are the basis of a list foradding terms to the vocabulary.

[0090] Tools to manage vocabulary include:

[0091] A morphological analyzer. This tool strips off any endings andmorphological alterations in a query to find the stem, and checks to seeif the stem is in the current vocabulary. If the stem is not, the userdoing the maintenance might try:

[0092] A spell checker. This tool uses the conventional algorithms tosee if the supposedly new word is actually a misspelling of a knownword. If it is not a misspelling, the user might try:

[0093] A name identifier. This tool checks to see if the supposedly newword is in a name configuration, in that it follows a known first namein the query. If it does, it is added to a candidate name database. Ifit is not, it is proposed as a possible new word to be added to thesystem's vocabulary.

[0094] Searching Audio/Video by Timecode Correlation with SearchCriteria.

[0095] Video and audio files can be timecoded, or marked such that thesoftware in which they run can locate a specific frame (for videos) ormeasure (for audio) at any time. Importantly, the system according tothe invention permits searching timemedia, including video and audiofiles, by combining two search elements. The first is a standard search,including but not limited to natural language search. The second is atime indicator, such as a SMPTE (Society of Motion Picture andTelevision Engineers) standard timecode. Face recognition is anadditional technology that can be used in searching. Face recognition isa subset of the more general technology of object recognition, andindeed techniques described here may extend to additional technologiesas well.

[0096] The current state of the art in face recognition technology makesit possible to take a manually created, labeled library of faces, andmatch faces from a video to that library. For example, a user might workwith a news video and use a face recognition program to label NelsonMandela in it. The output of the face recognition program would be atime-coded segment, with start and stop times, of when Nelson Mandelawas on camera, with the label “Nelson Mandela” attached to the timecodes. While face recognition currently does not achieve 100% precisionor recall, it can still be useful. For example, one known system offersa contract rights management capability for films that demandstime-coded segments with names attached, and assumes that users willcreate those manually, so that the correct contract restrictions foreach film segment can be attached to the right time codes. Given a smalllibrary of the actors in a film, it would be possible to do a fast,automated match-up of time codes and actors, even with imperfect facerecognition technology. Selecting the correct actor from forty publicityshots would be much simpler than selecting from among thousands offaces.

[0097] Importantly, the system according to the invention carries outthe automated creation of the face library. Required elements includetime-coded metadata (for example, the voice recognition transcript of avideo), and the ability to find the names of people in text. Each time aface and a person's name appear at the same time code, that occurrenceis a potential new entry for the face library. A user may run thefacematcher for thousands of hours and sift out the recurring matches asthe most likely. In this way, a reference library of faces is created,and new material can be catalogued automatically.

[0098] The software according to the invention approaches this by usingalignment techniques to match up two or more streams of metadata. Forexample, a broadcast news program may contain closed captioning for thehearing-impaired. It may also contain a separate description of the newsfootage, probably created manually by the news department. The systemaccording to the invention uses alignment to match the description,which is not time-coded, with the closed captioning, which istime-coded. This process allows the system to add time codes to thenon-time-coded stream. The software then uses that new, derived stream(i.e. the description with newly added time codes), and searches forproper names within it. At the same time, using face recognitionalgorithms on the video stream, the software finds faces. The systemtries to match up the faces with the proper names that describe who theyare. This matched set provides us with a rough cut of a face (or object)reference library. This is exemplified in FIG. 4.

[0099] Face recognition can also be employed to manage the library orarchive of media files. Media libraries are assembled over time, oftenfrom disparate sources, and may contain multiple copies of a singlemedia file, either with the same metadata or with different metadata.Duplicate detection is therefore an important element of library andarchive management, and face recognition (and, more generally, imagerecognition) can be leveraged to provide that capability. More broadly,for video, scene detection technology can assist in the process ofidentifying duplicates so that they can be purged from the library.

[0100] Clustering and Other Ways to Determine Stored File Usage.

[0101] Clustering involves combining user search queries in such a waythat the searches can be analyzed usefully to provide answers tobusiness questions. Clustering has received considerable attention indocument information retrieval (IR) and more recently, in video IR as ameans of refining retrieval results based on user preferences orprofiles, and to characterize the marketplace. The prior art containsmany examples of clustering applied in information retrieval systems,but they all apply to search results returned to users rather thansearch queries submitted by users. In the system according to theinvention, we cluster search queries by topic. We then use thatinformation to adjust the collections of stored files so that the filecollections will better meet users' needs.

[0102] This system characterizes the information needs of groups (andsubgroups) of users, with respect to a collection of media files (e.g.images, videos, sound clips, text, multimedia objects). Some commongroupings include:

[0103] search queries that brought back no files

[0104] search queries that brought back no files the user was interestedin

[0105] search queries that lead to expressions of user interest or sales

[0106] This system applies clustering technology to user-submittedsearch queries, and to the files retrieved in search results. It alsoincludes:

[0107] machine learning as applied to the above.

[0108] characterizing the information needs over time, by user type, orby other factors.

[0109] methods for reporting file collection needs to interested parties(for example, media suppliers). The system informs a supplier thatpictures of earthquakes are selling briskly, or that users keep lookingfor videos of dance performances but cannot find any.

[0110] methods for adjusting file collections based on the results ofthe clustering analysis, above.

[0111] novel clustering techniques. These include using a semanticexpansion (such as the WordNet hierarchical thesaurus) and phraseidentification (such as noun phrase, and name and locationidentification) as the basis for the clustering.

[0112] Before user queries can be analyzed, they must be expanded to a“common denominator”. To expand the user search queries, we use naturallanguage techniques. Specifically, we treat each query as if it weremetadata within our system, as described in the NLP section, above. Eachquery is expanded through the application of a semantic net, so that itcontains synonyms and related terms. This expansion provides a set offeatures to which we can apply standard clustering technology.

[0113] Obtaining valuable information on user preferences for storedfiles begins with deciding what information a client wants tounderstand. The data set can be selected according to various criteria,including:

[0114] queries from a particular subset of users (e.g. registered users,users by industry, new users)

[0115] queries that lead to success (sale or other indication)

[0116] queries that lead to failure

[0117] A first step is to select the data set on which clustering is tobe performed. In an information retrieval (IR) context, clustering canbe performed on queries or on assets to be retrieved (documents, images,video, audio, mixed media). A sample query set may include shortqueries, as is standard on Web searches, long queries, as seen in TREC(U.S. government-sponsored text retrieval conferences), or as producedby QBE (query by example), in which an asset or set of assets arethemselves inserted into a query.

[0118] A second step is to perform analysis on the queries using, forexample, linguistic methods, such as:

[0119] Tokenization: determine word/token boundaries. In English,tokenization mostly coincides with spaces between words, with certaincomplications (Alzheimer's=1 token vs. she's=2 tokens).

[0120] Morphology or stemming: removed tense and plural markers andother affixes to find the word root.

[0121] Identify names, locations, noun phrases: using a pattern matcheror other methodology, determine words were groupings for specialhandling. For example, for names, match certain kind of variance; forlocations, match subset; for noun phrases, we to complete andheadmatches higher than modifier homematches.

[0122] A third step is to expand the queries. Ideally, this stepincludes expansion using a thesaurus or semantic net of synonyms,superand other relationships.

[0123] A fourth step is, for each of the terms in each expanded query,assign a weight based on how close that term is to the original query.The exact weighting will vary by application, but the basicunderstanding is that more closely matching terms are weighted close to100.

[0124] A fifth step is to create a vector for each expanded query. Inorder to apply a statistical clustering algorithm, we arrange thevectors into a matrix.

[0125] A sixth step is to apply a statistical clustering algorithm inorder to group similar queries together. A hierarchical or linearclustering strategy may be used, depending on whether the desiredclusters are hierarchical or not. Clustering may allow overlap, whichmeans that a query may appear in more than one cluster.

[0126] A seventh step is to apply the clustering algorithm until thestopping condition is met, e.g. the desired number of clusters isobtained, or a combination of cluster number and desired distinctivenessof clusters is reached.

[0127] An eighth step relates to the clusters. Clusters are most usefulto a human observer if they bear useful names that reflect the content.Use a semantic net hierarchy, combined with term frequency in areference corpus, to identify the lowestterm in the hierarchy thatsubsumes all the queries.

[0128] Folding Popularity Into Rankings.

[0129] Many information retrieval applications currently incorporaterelevance feedback into their judgements of how to rank search resultsreturned to a user. In all cases, however, the past systems utilizeexplicit user feedback, not implicit feedback. That is, they rank filesby requiring a user to indicate what items he is interested in, once aset of items is returned by a search. Importantly, in the systemaccording to the invention, the system discerns implicit popularityrankings based on a ranked set of user actions. The system then thenuses those rankings to resubsequent search results. The user actionsfrom which popularity may be determined include, but are not limited to:

[0130] whether a file is placed in a projects folder or other work space

[0131] whether a file is placed in a shopping cart

[0132] whether a file is purchased

[0133] In addition, implicit popularity rankings may be derived from oneuser or set of users and applied to a different user or set of users.For example, if User A places a media file in a projects folder orshopping cart, the information on her activity can also be used toresearch results for User B, who is in some way similar to user A. Inanother example, if users with “.edu” e-mail addresses buy certainthings, it makes sense to research results to favor those things whenshowing results to other “.edu” users. In the system according to theinvention, if registered users who work for advertising agencies haveplaced certain items in their shopping carts, other advertising agencyemployees can have their search results re-ranked to favor those items.The opposite can be true as well: the same system can be used todisfavor certain items because they have been sold too many times toadvertising agencies, for example.

[0134] Retrieval System Self-Evaluation.

[0135] The retrieval system according to the invention does not answerany of the TREC tracks—a media file is described by a short paragraph orby keywords. Even though it is possible to develop a unique testcollection for purposes of evaluation, it will not necessarily predictthe performance of new systems, or even for existing ones.

[0136] What is now described is a method for ongoing evaluation of IRsystem performance based on search results combined with user feedback.This approach enables the system to alert a human system manager aboutobserved degradation in the system performance. As the performanceevaluation system becomes more knowledgeable through methods of machinelearning, the system is desirably able to change its own parameters inorder to improve its performance.

[0137] “Performance” as used in this inventive system is a score on ascale of user satisfaction with the respect to search results rankings.User satisfaction is determined by tracking the user actions togetherwith the system's data. For example, for a very good performance, theuser would submit a search query and purchase the media file that wasranked first. Possible indications of poor performance might include auser issuing several consecutive queries, getting thousands of results,and terminating the session without further examining any of the mediafiles.

[0138] Viewer.

[0139] The software application according to the invention includes aviewer for annotations, or mark-ups, of full motion digital stillimages, video frames, or other indexed data structures. The viewer,preferably accessible on a personal computer with a Web browser, servesas a helper application for the underlying application of the systemaccording to the invention. The viewer allows users to view and mark up(comment on) media on a separate, see-through lightweight layer. Theviewer supports annotations that are created and rendered so that theyappear overlaid on the related media. Position, size, frame number, andother attributes of annotations are saved in a separate annotation file,and when played back the annotations appear at the saved location andwith the saved attributes. Preferably, to the greatest extent possible,creating and editing of annotations is be done in WYSIWYG (what you seeis what you get) fashion. This is illustrated in FIG. 5.

[0140] The annotations may consist of text, graphics, and digitizedaudio, and are stored separately from the original file information. Anindexing scheme relates individual annotations to individual videoframes. On full motion video playback, the annotations are displayed onthe originally annotated frames. One central part of the systemaccording to the invention is the use of a downloadable, view-onlyapplication, which enables users who do not have the full power tocreate annotations nonetheless to be able to view annotations created byothers.

[0141] In an exemplary embodiment, files are stored centrally.Preferably the viewer provides security, e.g. SSL, serverauthentication, and proxy authentication. The underlying applicationaccording to the invention powers the full lifecycle of digital mediafrom content collaboration, deep storage, search and retrieval using anatural language search and the ability effectively to movebandwidth-intensive media over the Web. The software is preferablyWeb-based.

[0142] The viewer is a standalone, platform “helper” application that isnormally launched by a Web browser, from a specific link on a Web pagethat is part of the Web server of the system according to the invention.The viewer has the ability to upload annotation data back to the Webserver. Except for the two actions of launching and uploading, theviewer does not require any network connection with the Web server orthe Web browser.

[0143] The current viewer in the system according to the invention on aPC or Macintosh platform allows the user to view and annotate variousvideo, still image, audio, and text formats, including MPEG, QuickTime,or RealMedia videos, bitmap, jpeg, and tiff images, MP3, .wav, .aiff,.au, and .snd audio files, and .pdf document files. The viewer currentlyhas nine modes in which it can operate, all of which would be relevantto the mobile user. They are:

[0144] annotate a media file

[0145] select a keyframe

[0146] select storyboard frames (and optionally a keyframe)

[0147] set the download (cache) directory

[0148] send a message and, optionally, attached files to another user

[0149] download a media file

[0150] upload media files to a project

[0151] upload media files to a public collection

[0152] select a playlist

[0153] The current (Mac and Windows) viewers can be launched in any ofthese nine modes, generally by receiving an XML annotation file.

[0154] Attributes common to all annotation types are: position, size,frame number (video only), and page number (multipage document only).All of the graphical attributes such as colors and fonts are modifiableusing platform-native dialog boxes. These attributes can be set for newannotations as well as previously created ones.

[0155] For video, each annotation is recorded with the timecode of thespecific frame on which it was created. When movies with annotations areplayed back, the viewer stops playback at each frame where an annotationexists. The viewer also contains a droplist of all annotated frames (bytimecode) so that individual annotated frames can be viewed quickly.Controls are also provided for stepping by a single frame in eitherdirection, as well as a control displaying the timecode of the currentframe where the viewer is paused.

[0156] PDA (or handheld or wearable) version of the viewer builds on thesame general architecture. The server generates a simple XML annotationsfile, sends it to the client (i.e. the viewer via the Web browser), andhandles the returned HTTP messages from the viewer.

[0157] The following annotation types are desirably supported on a PDAversion:

[0158] Annotation Types Type Attributes Freehand drawing Line width,Color (pen) Transparent text Font name, Font size, Font color Stickynote Font name, Font size, Font color, Background color Sound Audio data

[0159] In this way, the system according to the invention supports thefull life cycle of media; allowing organizations to expedite the processof searching, collaborating and distributing of digital assets, such asimagery, video files, pictures and text documents. Users desirably havethe freedom to collaborate anywhere at anytime using the annotationviewer according to the invention on a PDA. Users are preferably able toannotate video and still images with a virtual pen, sticky note, textand even voice using a wireless modem. Annotations are preferablycapable of being added, edited, moved, or deleted.

[0160] A PDA-based digital media management solution takes digitaldelivery and collaboration to the next level by eliminating the need ofa traditional PC and Ethernet connection. In addition, users no longerneed to carry laptops to stay up to date on work involving video orstill images.

[0161] Preferably the viewer supports both HTTP authentication and Proxyauthentication. As an option, the viewer displays a login dialog wherethe user can enter a user ID and password to be used for authentication.Since the viewer is activated and closed frequently, it should notprompt the user to log in each time it connects to the server. Theviewer instead provides a mechanism for saving the user ID and passwordon the local workstation (in encrypted form) and passing thisinformation to the server without any prompting. It is desirable thatthe viewer support SSL encryption for all of its HTTP requests.

1. A method for use with first and second metadata streams with respectto a video stream, the first metadata stream time-coded with respect tothe video stream and the second metadata stream not time-coded withrespect to the video stream, the method comprising the steps of:aligning the second metadata stream with the first metadata stream;adding time codes to the second metadata stream, based on the alignment;searching for proper names within the second metadata stream; findingfaces within the video stream; matching faces with proper names; andplacing the matched faces and proper names into a reference library. 2.The method of claim 1 wherein the video stream is news footage, whereinthe first metadata stream is closed captioning for the hearing impaired,and wherein the second metadata stream is a separate description of thenews footage.
 3. A method for use with a system storing myriad digitalmedia files, the method comprising the steps of: generating myriad firstthumbnail images, each first thumbnail image corresponding to one of thedigital media files, each first thumbnail image having lower resolutionand smaller dimensions then the corresponding digital media file, eachfirst thumbnail image having first dimensions; displaying a subset ofthe first thumbnail images, the subset comprising more than one and lessthan all of the myriad first thumbnail images, the subset created as aresult of input from a first user; receiving a configuration commandfrom a second user, the configuration command indicative of seconddimensions differing from the first dimensions; generating myriad secondthumbnail images, each second thumbnail image corresponding to one ofthe digital media files, each second thumbnail image having lowerresolution and smaller dimensions then the corresponding digital mediafile, each second thumbnail image having second dimensions; displaying asubset of the second thumbnail images, the subset comprising more thanone and less than all of the myriad second thumbnail images, the subsetcreated as a result of input from a third user.
 4. A method for use witha system storing myriad digital media files, the method comprising thesteps of: generating myriad first thumbnail images, each first thumbnailimage corresponding to one of the digital media files, each firstthumbnail image having lower resolution and smaller dimensions then thecorresponding digital media file, each first thumbnail image havingfirst dimensions; displaying a subset of the first thumbnail images, thesubset comprising more than one and less than all of the myriad firstthumbnail images, the subset created as a result of input from a firstuser; a configuration command from a second user, the configurationcommand indicative of second resolution differing from the firstresolution; generating myriad second thumbnail images, each secondthumbnail image corresponding to one of the digital media files, eachsecond thumbnail image having lower resolution and smaller dimensionsthen the corresponding digital media file, each second thumbnail imagehaving second resolution; displaying a subset of the second thumbnailimages, the subset comprising more than one and less than all of themyriad second thumbnail images, the subset created as a result of inputfrom a third user.
 5. A method for use with a system storing digitalmedia records, the system comprising a vocabulary file of words keyed tothe digital media records, the method comprising the steps of: receivinga search query by computer from a user, the search query includingwords; logging the search query by computer, yielding a query log;processing the query log to identify words in the query log which arenot words already included in the vocabulary file and which are notwords which are variants of words already included in the vocabularyfile, the processing performed by computer, said words defined as“not-found words;” adding the not-found words to the vocabulary file,and the not-found words to the digital media records.
 6. A method foruse with a system storing digital media records, the system comprising adata file defining rights management questions and multiple-choiceanswers thereto, the method comprising the steps of: receiving, from afirst user, a selection of rights management questions and respectivemultiple-choice answers for the data file; receiving, from a second usernon-identical to the first user, a query regarding a first digital mediarecord; presenting, to the second user, a first rights managementquestion among the selection of rights management questions; receiving,from the second user, a multiple-choice answer from among the respectivemultiple-choice answers with respect to the first rights managementquestion, and storing same; presenting, to the second user, a secondrights management question among the selection of rights managementquestions; receiving, from the second user, a multiple-choice answerfrom among the respective multiple-choice answers with respect to thesecond rights management question, and storing same; presenting, to thesecond user, a third rights management question among the selection ofrights management questions; receiving, from the second user, amultiple-choice answer from among the respective multiple-choice answerswith respect to the third rights management question, and storing same;deriving pricing information for the second user with respect to thefirst digital media record from the stored multiple-choice answers;receiving, from a third user non-identical to the first user andnon-identical to the second user, a query regarding a second digitalmedia record; presenting, to the third user, a fourth rights managementquestion among the selection of rights management questions; receiving,from the third user, a multiple-choice answer from among the respectivemultiple-choice answers with respect to the fourth rights managementquestion, and storing same; presenting, to the third user, a fifthrights management question among the selection of rights managementquestions; from the third user, a multiple-choice answer from among therespective multiple-choice answers with respect to the fifth rightsmanagement question, and storing same; presenting, to the third user, asixth rights management question among the selection of rightsmanagement questions; receiving, from the third user, a multiple-choiceanswer from among the respective multiple-choice answers with respect tothe sixth rights management question, and storing same; deriving pricinginformation for the third user with respect to the second digital mediarecord from the stored multiple-choice answers.
 7. A method for use witha system storing digital media records, the system comprising a searchengine the operation of which is defined by a plurality of storedparameters, the stored parameters defining, for the search engine, anextent to which the search engine expands search terms, the systemcommunicatively coupled to an internet and providing a hypertexttransfer protocol server interface to the internet, the methodcomprising the steps of: receiving from a first user, via a hypertexttransfer protocol client session, a change to one of the parameters;storing the change to the one of the stored parameters; from a seconduser, via a hypertext transfer protocol client session, a searchrequest; passing the search request to the search engine; performing, bythe search engine, a search determined by the search request anddetermined by the stored parameters, and deriving search resultstherefrom; reporting the search results to the second user.
 8. A methodfor use with a system storing digital media records, the systemcomprising a search engine searching said digital media records, themethod comprising the steps of: displaying all or part of a firstdigital media record to a user; receiving, from the user, firstinformation indicative of a selection of less than all of the displayedportion of the first digital media record; receiving, from the user,second information indicative of text; performing, by the search engine,a search of the digital media records with respect to the firstinformation and the second information, and deriving search resultstherefrom; and displaying the search results to the user.
 9. A methodfor use with a system storing digital media records and comprising asearch engine searching said stored digital media records, the methodcomprising the steps of: receiving search requests from users; loggingthe search requests; expanding the search requests; applying astatistical clustering algorithm, thereby grouping similar searchrequests together; identifying, using a semantic net hierarchy, alowest-level term in the hierarchy that subsumes all of the queries in agrouping of search requests; communicating the identified term to auser.
 10. A method for use with a system storing digital media recordsand comprising a search engine searching said stored digital mediarecords, the method comprising the steps of: receiving search requestsfrom users; by the search engine, searches based upon the searchrequests, yielding respective search results, each search resultdefining selected digital media records, the digital media records eachhaving associated metadata; logging the search results and the metadataassociated with digital media records selected therein; expanding themetadata, defining expanded query metadata results; applying astatistical clustering algorithm, thereby grouping similar expandedquery metadata results together; identifying, using a semantic nethierarchy, a lowest-level term in the hierarchy that subsumes all of theexpanded query metadata results in the grouping of expanded querymetadata results; communicating the identified term to a user.
 11. Amethod for use with a system storing digital media records andcomprising a search engine searching said stored digital media records,the method comprising the steps of: receiving first search requests froma plurality of first users; performing, by the search engine, searchesbased upon the first search requests, yielding respective first searchresults, each first search result defining first selected digital mediarecords; the first search results; receiving, from the first users,information indicative of subsequent actions by the first usersselecting particular ones of the selected digital media records;receiving a second search request from a second user; performing, by thesearch engine, a search based upon the second search request, yieldingrespective second search results, the second search results definingsecond selected digital media records; ranking the second selecteddigital media records downward for at least one second selected digitalmedia record that was selected by at least one first user.
 12. A methodfor use with a system storing digital media records and comprising asearch engine searching said stored digital media records, the methodcomprising the steps of: receiving first search requests from aplurality of first users; performing, by the search engine, searchesbased upon the first search requests, yielding respective first searchresults, each first search result defining first selected digital mediarecords; logging the first search results; receiving, from the firstusers, information indicative of subsequent actions by the first usersselecting particular ones of the selected digital media records;receiving a second search request from a second user; performing, by thesearch engine, a search based upon the second search request, yieldingrespective second search results, the second search results definingsecond selected digital media records; ranking the second selecteddigital media records downward for at least one second selected digitalmedia record that was selected by at least a predetermined number offirst users.
 13. A method for use with a system storing digital mediarecords and comprising a search engine searching said stored digitalmedia records, the method comprising the steps of: receiving firstsearch requests from a plurality of first users; performing, by thesearch engine, searches based upon the first search requests, yieldingrespective first search results, each first search result defining firstselected digital media records; logging the first search results;receiving, from the first users, information indicative of subsequentactions by the first users selecting particular ones of the selecteddigital media records; receiving a second search request from a seconduser; performing, by the search engine, a search based upon the secondsearch request, yielding respective second search results, the secondsearch results defining second selected digital media records; rankingthe second selected digital media records downward for at least onesecond selected digital media record that was selected at least apredetermined number of times by first users.
 14. A method for use witha system storing digital media records and comprising a search enginesearching said stored digital media records, the method comprising thesteps of: first search requests from a plurality of first users;performing, by the search engine, searches based upon the first searchrequests, yielding respective first search results, each first searchresult defining first selected digital media records; displaying saidfirst selected digital media records presented in an order, said orderdefining a first-presented first selected digital media record; loggingthe event of a first user purchasing said first-presented first selecteddigital media record; reporting, to a second user, the frequency ofoccurrence of the logged events.
 15. A method for use with a systemstoring digital media records for purchase and comprising a searchengine searching said stored digital media records, the methodcomprising the steps of: receiving first search requests from aplurality of first users; performing, by the search engine, searchesbased upon the first search requests, yielding respective first searchresults, each first search result defining first selected digital mediarecords; displaying said first selected digital media records presentedin an order, said order defining a first-presented first selecteddigital media record; logging the event of a first user issuing severalconsecutive search requests, and receiving respective first selecteddigital media requests greater in number than a predetermined thresholdfor said several consecutive search requests; to a second user, thefrequency of occurrence of the logged events.
 16. A method for use witha system storing digital media records and comprising a search enginesearching said stored digital media records, the method comprising thesteps of: receiving search requests from users; performing, by thesearch engine, searches based upon the search requests, yieldingrespective search results, each search result defining selected digitalmedia records or being empty; logging the search requests for which thesearch result is empty; expanding the logged search requests; applying astatistical clustering algorithm to the logged search requests, therebygrouping similar search requests together; communicating a group ofsearch requests to a user.
 17. A method for use with a system storingdigital media records and comprising a search engine searching saidstored digital media records, the system permitting user expressions ofinterest in particular stored digital media records, the methodcomprising the steps of: search requests from users; performing, by thesearch engine, searches based upon the search requests, yieldingrespective search results, each search result defining selected digitalmedia records; logging the search requests for which a user hasexpressed interest in a selected digital media record; expanding thelogged search requests; applying a statistical clustering algorithm tothe logged search requests, thereby grouping similar search requeststogether; communicating a group of search requests to a user.
 18. Amethod for use with a system storing digital media records andcomprising a search engine searching said stored digital media records,the method comprising the steps of: receiving first search requests froma plurality of first users; performing, by the search engine, searchesbased upon the first search requests, yielding respective first searchresults, each first search result defining first selected digital mediarecords; logging the first search results; receiving, from the firstusers, information indicative of subsequent actions by the first usersexpressing interest in particular ones of the selected digital mediarecords; receiving a second search request from a second user;performing, by the search engine, a search based upon the second searchrequest, yielding respective second search results, the second searchresults defining second selected digital media records; ranking thesecond selected digital media records upward for at least one secondselected digital media record that was for which at least one first userexpressed interest.
 19. A method for use with a system storing digitalmedia records and comprising a search engine searching said storeddigital media records, the method comprising the steps of: receivingfirst search requests from a plurality of first users; performing, bythe search engine, searches based upon the first search requests,yielding respective first search results, each first search resultdefining first selected digital media records; logging the first searchresults; receiving, from the first users, information indicative ofsubsequent actions by the first users expressing interest in particularones of the selected digital media records; receiving a second searchrequest from a second user; performing, by the search engine, a searchbased upon the second search request, yielding respective second searchresults, the second search results defining second selected digitalmedia records; ranking the second selected digital media records upwardfor at least one second selected digital media record that was for whichat least a predetermined number of first users expressed interest.
 20. Amethod for use with a system storing digital media records andcomprising a search engine searching said stored digital media records,the method comprising the steps of: receiving first search requests froma plurality of first users; performing, by the search engine, searchesbased upon the first search requests, yielding respective first searchresults, each first search result defining first selected digital mediarecords; logging the first search results; receiving, from the firstusers, information indicative of subsequent actions by the first usersexpressing interest in particular ones of the selected digital mediarecords; receiving a second search request from a second user;performing, by the search engine, a search based upon the second searchrequest, yielding respective second search results, the second searchresults defining second selected digital media records; ranking thesecond selected digital media records upward for at least one secondselected digital media record for which first users expressed interestat least a predetermined number of times.
 21. A method for use with asystem storing digital media records and comprising a search enginesearching said stored digital media records, the method comprising thesteps of: receiving first search requests from a plurality of firstusers; performing, by the search engine, searches based upon the firstsearch requests, yielding respective first search results, each firstsearch result defining first selected digital media records; logging thefirst search results; receiving, from the first users, informationindicative of subsequent actions by the first users expressing interestin particular ones of the selected digital media records; receiving asecond search request from a second user; performing, by the searchengine, a search based upon the second search request, yieldingrespective second search results, the second search results definingsecond selected digital media records; the second selected digital mediarecords upward for at least one second selected digital media record forwhich expression of interest satisfies a predetermined threshold,wherein expression of interest comprises a weighted function of two ormore of the following events: the user placing the selected digitalmedia record into an online shopping cart, the user purchasing rights touse the selected digital media record, and the user placing the selecteddigital media record into an online projects folder or other work space.22. A method for use with a metadata stream with respect to a videostream, the metadata stream time-coded with respect to the video stream,the method comprising the steps, performed by a computer, of: searchingfor proper names within the metadata stream; finding faces within thevideo stream; matching faces with proper names; and placing the matchedfaces and proper names into a reference library.
 23. The method of claim22 wherein the video stream is news footage, wherein the metadata streamis closed captioning for the hearing impaired.
 24. A method for use witha metadata stream with respect to at least one video stream, themetadata stream time-coded with respect to the at least one videostream, the method comprising the steps, performed by a computer, of:searching for proper names within the metadata stream; finding faceswithin the at least one video stream; matching faces with proper names;the matched faces and proper names into a reference library; and for aselected proper name, generating a list of locations within the at leastone video stream associated with that selected proper name.
 25. Themethod of claim 24 wherein the video stream is news footage, wherein themetadata stream is closed captioning for the hearing impaired.
 26. Themethod of claim 24 further comprising the step of communicating the listof locations to a rights management system.
 27. A method for use with asystem for managing digital media files, the method comprising the stepsof: using face recognition to recognize faces-portrayed in the digitalmedia files, yielding metadata with respect to the digital media filesindicative of the recognized faces; analyzing the metadata indicative ofthe recognized faces to detect duplicate files among the digital mediafiles; and displaying the duplicate files for a user.
 28. A method foruse with a system for managing digital media files, the methodcomprising the steps of: using image recognition to recognize imagesportrayed in the digital media files, yielding metadata with respect tothe digital media files indicative of the recognized images; analyzingthe metadata indicative of the recognized images to detect duplicatefiles among the digital media files; and the duplicate files for a user.29. A method for use with a system for managing digital media files, themethod comprising the steps of: using scene detection to detect scenesportrayed in the digital media files, yielding metadata with respect tothe digital media files indicative of the detected scenes; analyzing themetadata indicative of the detecting scenes to detect duplicate filesamong the digital media files; and displaying the duplicate files for auser.